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New Learning: The Next Generation of Evaluation? 



Diane D. Chapman, Colleen Aalsburg Wiessner, Julia Storberg-Walker, and Tim Hatcher 
North Carolina State University 

New Learning is an innovative process, aimed at collaborative learning in professional and scholarly 
events, that we suggest may be the ‘next generation ’ of evaluation. In this manuscript we review the 
literature of evaluation, describe what new learning is and how it works, and then compare and contrast it 
with traditional evaluation methods and theory. We conclude with implications for future application and 
research. 
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Problem Statement 

Most professional eonferenees bring people together, yet they foeus primarily on sharing information rather than 
taking that information to the next level - new knowledge generation. Individuals typieally leave eonferenees with 
their own learning — yet often they do not benefit from the learning of others. Nor do they gain an understanding of 
the overall learning that took plaee at the event and that potentially eould benefit their field as a whole. To overeome 
these typieal limitations, we ereated what we suggest may be the ‘next generation’ of evaluation and eall it New 
Learning (NL). Embedded in eonferenee aetivities, NL took the plaee of a more traditional evaluation seheme. 

In plaee of standard end-of-session evaluation forms foeusing on presentations and presenters, partieipants 
submitted NL forms; aeeounts of what was aetually learned. We thought these innovations made sense: a researeh 
eonferenee should not refleet on the past solely, but generate new ways of thinking for the future as well. We agreed 
with Graham and Kormanik (2004) that too often eonferenees rely on one-way eommunieation with little time spent 
on diseussion or on ways to use information in future theory, researeh, and praetiee. 

Evaluation at professional eonferenees has historieally foeused on partieipant satisfaetion and reaetions to the 
event. A Web seareh for conference evaluation demonstrates foeus on quantitative results and partieipant reaetions. 
For example, the Ameriean Evaluation Assoeiation’s (Ameriean Evaluation Assoeiation, n.d.a) 2003 annual 
eonferenee foeused on overall eonferenee satisfaetion, with speeial emphasis on Presidential Strand and Plenary 
sessions, reasons why partieipants return, and eonferenee program materials. The 2005 Edueational Fund 
International Conferenee foeused on satisfaetion with sessions, relevaney of sessions, and reasons for attendanee 
(Results, n.d.a). The 2003 International Symposium on HIV/AIDS Workplaee Polieies and Programmes in 
Developing Countries evaluation foeused on whether or not sessions were “good,” areas for improvement, 
identifying issues to learn more about, and reeommendations (Joint Bank/Fund Health Serviees Department, n.d.a). 

When we look at stated purposes of eonferenees, we see diseonneets between purpose, theory, and method of 
typieal eonferenee evaluations. For example, AEA’s purpose was to “weleome evaluation praetitioners, aeademies, 
and students from aeross the US and around the world to learn from each other in a supportive atmosphere” 
(Ameriean Evaluation Assoeiation, n.d.b; italies ours). Joint Bank Fund Heath Serviees Department Symposium’s 
objeetives were to “share experienee with others, identify key faetors for developing best praetiees, pool teehnieal 
resourees and researeh, and develop networks with eolleagues who faee similar issues” (n.d.b; italies ours). The 
Edueation Fund Conferenee’s purpose was for partieipants to be “briefed in the latest information by experts and get 
trained in powerfully speaking the issues,” (Results, n.d.b; italies ours). All of these organizations hold eonferenees 
and symposia foeused on learning, yet their evaluations foeus primarily partieipant reaetion. This foeus is no 
different within most organizations, ineluding the Aeademy. We saw an opportunity to more explieitly link learning 
objeetives of eonferenees with evaluation, and ereated an approaeh we eall New Learning. 

The problem as illustrated is that eonferenees 1) say they want to ereate spaees for learning but do not evaluate 
learning that aetually does oeeur; and 2) use evaluation teehniques that aggregate individual responses, thereby 
potentially miss opportunities for finer grained analyses and diverse input. This manuseript ineludes four seetions. 
First, we deseribe the problem and researeh question. Seeond, we explain our theoretieal framework and literature 
review. Third, we present results and findings, eomparing evaluation with aspeets of NL. Fourth, we present 
eonelusions and reeommendations for future researeh and applieation of NL methods. 
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Research Questions 

The questions that guide this researeh are: What are the similarities and differenees between evaluation and NL? 
What eontributions does NL make to evaluation literature? We explore these questions in a eomparative literature 
review (Torroeo, 2005) and through review of an exploratory ease study eondueted at the 2005 AHRD International 
Researeh Conferenee held in Estes Park, CO (Hateher, Wiessner, Storberg-Walker, Chapman, 2005). 

Theoretical Framework and Literature Review 

To answer our researeh question, we looked at two areas of literature. One relates to learning and the seeond relates 
to evaluation. We begin with the learning literature that forms the foundation for NL. 

Collaborative Learning 

New Learning is a proeess by whieh we ean eapture knowledge ereation. This process of NL stemmed from 
earlier work at three International Conferenees on Transformative Learning, held at Teaehers College, Columbia 
University, in 1998, 2000, and 2003. These eonferenees were designed and implemented as eollaborative inquiries 
intended to ereate and eapture learning in event eontexts in order to build theory (Wiessner & Mezirow, 2000; 
Wiessner, 2004). Foundations of NL are based on the literature in the areas of learning organizations, eommunities 
of praetiee, and knowledge ereation. These three foundations were seleeted through a dialogie proeess eondueted 
over time between the four researehers presenting this manuseript. 

Learning Organizations 

Learning organization literature relates to NL for three reasons. First, this literature eaptures a primary mission 
of AHRD, and many other organizations, by providing details about roles of dialogue and sharing of ‘mental 
models’ in the proeess of learning (Senge, 1990). Both formal and informal nature and eontexts of learning are 
reeognized. Seeond, it enhanees the learning paradigm of HRD through both its eoneeptual and empirieal findings 
(see, for example, Watkins & Marsiek, 1993). Continuous learning and eapturing and eodifying learning, two 
eharaeteristies of organizations engaged in learning, shape this researeh. Third, the learning organization eoneept 
mirrors a goal of AHRD to support learning as a part of its proeesses and outeomes. Inquiry, eritieal refleetion, 
dialogue, and systems thinking, are proeesses that are valued in organizational learning and e key to NL approaehes. 
Learning oeeurs on individual, group, organizational and soeietal levels, yet learning beyond the individual level is 
not always realized (Kasl, Marsiek & Deehant, 1997) and groups do not always benefit from what individuals learn. 
The organizational learning literature helps to bridge that gap, as applied in NL; faeilitating shared learning aeross 
several levels “in ways that enable the larger system to learn” (Marsiek, Bitterman & Van der Veen, 2000, p. 13). 
Communities of Practice 

Communities of praetiee are defined as groups of people bound together by shared expertise and passion for a 
joint enterprise (Wenger & Snyder, 2000) or “a group of people who share a eoneem or a passion for something they 
do and who internet regularly to learn how to do it better” (Wenger, 2004). New Learning studies groups, sueh as the 
AHRD eonferenee, with three eomponents: bounded population, shared expertise, and regular interaetion to learn. 
Knowledge Creation 

Aeeording to Von Krogh, lehijo, and Nonaka (2000), new knowledge ereation involves five steps: sharing taeit 
knowledge, ereating eoneepts, justifying eoneepts, building prototypes, and eross-leveling knowledge. In addition, 
Nonaka and Toyama (2003) refer to SECI, four stages for knowledge ereation: soeialization, extemalization, 
eombination, and internalization. Of eritieal importanee to these knowledge generation proeesses is making taeit 
knowing explieit, based in the work of Polanyi (1973). Both new knowledge and new ways of creating knowledge 
are needed in our work and soeietal eontexts (Nonaka & Toyama, 2003). Of partieular interest in knowledge 
ereation is Nonaka, Toyama and Konno’s (2000) eoneept of 6a; a movable spaee ereated for learning that Storberg- 
Walker (2003) posits as the missing link between individual and organizational learning. 

Evaluation 

A literature review foeused on evaluation was eondueted for this manuseript. This seetion presents a broad 
speetmm of evaluation theories, elassifieations, and definitions in order to assess eongmeney between evaluation 
and New Learning. As we demonstrate below, there exists a eomplieated yet innovative relationship between 
traditional evaluation and the NL approaeh. Consequently, we suggest NL may be aeeurately deseribed as the ‘next 
generation’ of evaluation, as it builds upon existing evaluative frameworks and eoneepts while moving into realms 
of ereating and identifying future individual, organizational, and diseiplinary benefits. 

Definitions 

Merriam- Webster’s dietionary defines evaluation in two ways: 1) to determine or fix the value of and 2) to 
determine the signifieanee, worth or eondition of, usually by eareful appraisal and study (2004). Note that while the 
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first definition suggests a determination of positive or negative affect, the second embraces the idea of determination 
of condition. New Learning fits within this definition, removing the requirement of assigning value or worth. 

The American Evaluation Association provides a list of interests the field encompasses, such as evaluation of 
programs, products, personnel, policy, performance, proposals, technology, research, theory, and evaluation itself 
(American Evaluation Association, 2004). New Learning fits into several of these interest areas. Initially New 
Learning seems like an alternative approach applicable to determine the condition of program, performance, and 
research and theory. Delving further into evaluation literature clarifies where NL’s positionality. 

Defining Evaluation by Theory and Theoretical Foundations 

There is a cadre of researchers who have attempted to define evaluation through theory. Many do so by 
breaking up the field into realms associated with the use or purpose of the evaluation. We present seven alternative 
perspectives on the foundations of evaluation below: 

According to Alkin and Christie (2004), evaluation theory is built upon foundations of accountability and social 
inquiry. Accountability and need for control provide the rationale and social inquiry provides the methods. 
Evaluation models and theories have been categorized using a variety of schema. Alkin & Christie (2004) portray 
evaluation theories as falling in one of three branches of the same tree: methods, use, and valuing. Methods theories, 
guided by research methods, focus on knowledge construction. The valuing branch models focus on placing value 
on data. The third branch, use, represents models oriented toward decision-making. 

Scriven (2004) referred to evaluation as formative, done during development of the program, and summative, 
done at conclusion of the program. He later added the term ascriptive to cover those evaluations not tied to the 
program development process. Rossi (2004) contends evaluation is applied social research, listing three components 
of this view: 1) evaluation is social research applied to answering policy-oriented questions, 2) the primary aim of 
evaluation is to help stakeholders in their decision-making, and 3) evaluation involved making judgments. 

Chen defines evaluation by the following characteristics: future directedness, scientific and stakeholder 
credibility, and holistic approaches (2004). Greene sorts evaluation approaches by whose values are being promoted. 
Her four clusters are those serving 1) efficiency interests of policy-makers, 2) accountability and ameliorative 
interests of on-site program managers, 3) understanding and development interests of direct service staff and 
affiliates, and 4) democratic and social change interests of program beneficiaries and their allies (2004). 

Guba and Lincoln (n.d.) divide evaluation into four realms: logical positivism, based on belief that knowledge is 
based solely on observable facts; postpositivism, belief that observation is theory-laden and that research is 
influenced by theory; pragmatism, which places emphasis on the notion that truth is thought to be "what works for 
now", and constructivism, based on belief that knowledge depends on knowers’ frames of reference. 

Reeves (1997) proposed four evaluation paradigms. The positivist-quantitative paradigm relies on the 
quantitative notion of reducing situations to operationalized variables. The constmctivist-interpretive-qualitative 
paradigm views the world as socially constructed and focuses on social interactions and multiple perspectives. The 
critical theory-postmodem paradigm rejects positivistic approaches and looks not only at describing the world, but 
also changing it using theory and reflection. Finally, the eclectic-mixed methods-pragmatic paradigm involves 
complex evaluation strategies, both quantitative and qualitative and focuses on practical problems. 

Cervero and Wilson (1994) slice the evaluation pie in another direction. They distinguish three program 
evaluation viewpoints: classical, naturalistic and critical. The classical viewpoint, based on Tyler (1949), is highly 
scientific in nature and proposes four questions pertaining to purpose, content, methods, and evaluation. The 
naturalistic viewpoint, more situation-specific, uses practical reasoning. “Rather than applying standard principles, 
the naturalistic viewpoint places primary emphasis on the planner’s ability to make judgments in a specific context 
and to justify them” (Cervero & Wilson 1994, p. 18). In the critical viewpoint, characterized by concern for social 
and political emancipation, “Planners can make the best judgments in everyday practice... only if they clearly 
understand that education is a political and ideological activity intimately connected with the social inequalities of 
society as a whole” (Cervero & Wilson 1994, p. 21). 

Feminist Evaluation suggests that evaluation must do more that judge merit of a program; viewing it as a venue 
for increasing social justice for women (Patton, 2002). It is based on six main characteristics and concepts: 1) A 
central focus on the gender inequities that lead to social injustice, 2) Discrimination or inequality based on gender is 
systemic and structural, 3) Evaluation is a political activity, 4) Knowledge is a powerful resource, 5) Knowledge and 
values are culturally, socially, and temporally contingent and 6) There are multiple ways of knowing; some ways are 
privileged over others (Sielbeck-Bowen, Brisolara, Seigart, Tischler, & Whitmore, 2002). 

Clearly, in the terms of Aiken and Christie (2004), Scriven (2004), and Chen (2004), New Learning is social 
research but it is not focused on policy questions or decision-making, although it can be used for both. NL is more 
about making information visible and accessible, than about making judgments. It is not focused on efficiency or 
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accountability interests, although it does serve that purpose by consequence, and definitely serves interests of staff, 
affiliates and democratic and social change. NL fits into constructivism (Guba and Lincoln, n.d.). 

While NL definitely promotes change, using theory and reflection, it would most reasonably fall into the 
eclectic paradigm (Reeves, 1997) which incorporates a variety of strategies. New learning fits primarily within the 
critical viewpoint as described by Cervero and Wilson (1994). With NL planners forgo rigidity of scientific 
approaches and control of naturalistic approaches in favor of participant freedom valued in the critical approach. NL 
is concerned with social inequities and promotes ways of engaging marginalized voices and perspectives. 
Additionally, NL can be considered a form of feminist evaluation because it is structured so that all are heard. The 
single, individual voice is given as much attention as the voice of the majority. New Learning does not seek to 
explore and identify consensus opinion, rather all opinion. 

The seven perspectives discussed here define evaluation by its theoretical foundations and intent. New learning 
fails to fit neatly into any evaluation category when delineated by theoretical foundation, as it spans across these 
mutually exclusive buckets. The next section of this paper looks at how NL compares with evaluation perspectives 
that are based on characteristics of evaluation. 

Defining Evaluation by Characteristics of Evaluation 

This section presents five additional ways that traditional evaluation has been characterized in the evaluation 
literature. As demonstrated below, there are many aspects of traditional evaluation that represent the theoretical 
approach of NL. Distinctions and similarities emerge in the following discussion. 

Weiss (1972) provided characteristics of evaluation that distinguish it from other forms of research. Intent lies 
at the core. In contrast to other forms of research evaluation 1) is used for decision-making, 2) asks questions that 
are derived from the program and not from the evaluator, 3) involves judgment, 4) takes place in an action setting 
and 5) is wrought with role conflicts between evaluators and practitioners. Additionally, evaluation research is 
seldom published and evaluation researchers have dual if not triple allegiance. New Learning would therefore not 
constitute a form of evaluation by Weiss’ standards. NL is similar in that 1) it can be used for decision-making, 2) 
questions are developed from the program, not the evaluator, and 3) it takes place in an action setting. It differs in 
that NL is not necessarily wrought with conflict, does not involve judgment, evaluators may not have more than one 
allegiance, and publication is a goal of NL that also furthers organizational and disciplinary goals. 

Stufflebeam (2001) suggests “evaluation means a study designed and conducted to assist some audience to 
assess an object’s merit and worth.” He divided evaluation approaches into four categories, approaches that 1) 
promote invalid or incomplete findings, referred to as pseudoevaluations; 2) have questions or methods-oriented 
processes that address specified questions, either employing a wide range of methods or a particular method; 3) 
focus on improvement or accountability to fully assess a program’s merit and worth; and 4) focus on social agenda 
or advocacy approaches directed at making a difference in society through program evaluation. New Learning may 
fit in the category of method-oriented or question-oriented approaches, but the category as a whole does not 
seemingly encompass the intent of NL. 

Preskill’s & Torres’ (1999) work shifts the focus of evaluation from problem-focused to appreciative inquiry, 
or a focus on organizational successes. Aspects of their model include a reflexive, embedded nature of evaluation; a 
commitment to partnership between stakeholders and evaluators to apply findings; and an understanding that 
learning is an important goal of the evaluation process. Preskill and Torres’ (1999) EILO model provides guiding 
principles for NL, as NL mirrors appreciative inquiry by focusing on the future rather than emphasizing the past. 

Owen (2004) classifies evaluation by knowledge produced, although classifications also show stages in 
program development. He presents five evaluation types: 1) proactive synthesizes knowledge for the purpose of 
program design; 2) clarificative identifies aspects of a program to make them explicit to stakeholders; 3) 
participatory provides knowledge for continuous improvement; 4) monitoring makes sure a program is staying its 
course; and 5) impact evaluation determines effects of a program. NL contributes to all of these evaluation purposes. 

Watkins and Marsick (1993) look at learning at four levels, including individual, team, organization, and 
society. In fact, they further characterize organizational learning by a change in the organization’s capacity for 
innovation and new knowledge. NL encompassed both organizational and societal learning and innovation 
evidenced by benefiting the organization (e.g., AHRD) and the discipline of HRD as a whole. And, as described 
earlier, the organizational learning literature is one dimension of the conceptual framework for NL. 

Results and findings 

Synthesis of New Learning with Traditional Evaluation 

Because there are many ways of defining and categorizing evaluation, no one perspective, theory, or model 
provides an exact understanding of NL. We suggest that NL is accurately considered as the ‘next generation’ of 
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evaluation, and in this section we present characteristics of NL, comparing each to what we know about evaluation. 
Table 1 summarizes these similarities and differences. 



Table 1. Characteristics of New Learning and Evaluation 



Characteristic 


New Learning 


Evaluation 


Analysis of information 


Involves capturing and codifying 


Involves judging, valuing, or rating 


Comparative nature 


Is relative to the event 


Is comparative in nature 


Time (when) 


Focused on future 


Focused on past 


People (who) 


Focused on learner 


Focused on presenter 


Type of information 


Focused on unknown 


Focused on known 


Guiding questions 


Why and how 


Good or bad 


Impact of environment 


Globally situated 


Locally situated 


Breadth of analysis 


Holistic and on-going 


One-time snapshot 


Use of results 


Ability to build from one NL to another 


Usually limited to one use 


Use 


Change 


Maintain status quo 


Critical nature 


Constructive 


Judgmental 




Embracing 


Discriminatory 


Sources of information 


Focused on the planned and unplanned 


Focus on the planned 




Focused on the formal and informal 


Focused on the formal 


Audience 


Everyone 


Formal stakeholders 


Purpose 


What can we do? 


How did we do? 


Social implication 


Promotes collaboration 


Promotes us versus them 


Impact 


Aimed at field, profession, research, practice 


Aimed at program, event, presentation 




Revolutionary 


Evolutionary 



Evaluation involves judgment; rating or ranking of items according to value or importance. NL does not ask 
participants to rank or rate their learning - only to capture it. Evaluation is geared toward comparison. New 
Learning is relative to the event studied; rather than comparing, it tries to understand. Evaluation focuses on the past. 
New Learning focuses on the future. Evaluation captures what we already know. New Learning captures where we 
are going and future possibilities. Evaluation focuses on presenters; NL focuses on learners. Evaluation attempts to 
identify what was right or wrong; NL attempts to capture not only the “what” but also the “how” and “why”. 
Evaluation does not typically involve the environment, or systems thinking; NL considers the environment to 
include learning that occurs in all aspects of an event. Evaluation is usually a one-time picture of what happens at an 
event. New Learning presents the event holistically; including aspects of the event well after it is over. Results from 
one evaluation can rarely be used in the next. In NL, one can enhance and build on results from one event in future 
events. Evaluation is oriented toward maintaining the status quo. New Learning is oriented toward change. Initial 
findings, feedback on what was learned are incorporated into the conclusion of the event, another asset. 

Evaluation is static, whereas New Learning is dynamic. Evaluation is judgmental; NL is constructive. 
Evaluation is discriminating, predetermining which factors are important or not important ahead of time; NL is 
embracing, allowing for previously unidentified factors to emerge. Evaluation focuses on formal presentation, 
program, or events; NL includes everything that happens, acknowledging that every moment is a potential learning 
moment. Evaluation is for formal stakeholders; NL is for everyone. Evaluation tells us how we did and NL helps us 
figure out what we can do. Evaluation promotes an “us versus them” environment; NL is collaborative in nature. 
Evaluation is aimed at an event, presentation, or program; NL is aimed at the field, profession, theory or practice. 

Evaluation and NL share some similarities; both connect to mission and related outputs of systems under study. 
New Learning is best geared toward knowledge-intensive organizations, such as professional conferences, because 
mission related outputs of conferences are new learnings. New Learning is most appropriate where knowledge is a 
primary goal. New Learning, one way to evaluate, changes the nature of what evaluation can do for organizations. 
Discussion of New Learning In Practice: The 2005 AHRD International Research Conference 
Execution of NL at the 2005 AHRD conference was considered a case study because it was an exploratory process 
occurring in a single setting with a purposefully selected group of people. New Learning used both descriptive and 
qualitative data analysis methods (Stake, 1995) and satisfied the three requirements of qualitative research methods: 
describing, understanding, and explaining. “Qualitative data analysis was used to analyze attendee responses of 
‘what’ and ‘how’ they learned. Descriptive statistics were used to understand trends and differences in attendee 
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learnings” (Hatcher, et. al, 2005, p 3). Descriptive statistics included both individual and daily form submission 
counts and analysis of perceived professional status. 

AHRD organizational leaders supported the project and communicated support through conference literature, 
pre-conference announcements, and Academy publications. These communications were aimed at gamering 
attendee interest and participation in the process, and began three months before the event. In addition, AHRD 
distributed a white paper on NL as part of registration materials (Hatcher, et. al., 2005). 

At the conference, NL ‘05 was featured in every area of the conference. Red and white banners, signs, and staff 
T-shirts featured the light bulb logo selected for the event. A NL table sat next to the registration area and registrars 
directed participants there following registration. Each conference attendee received an overview of the project, an 
introduction to the data collection materials, and an invitation to participate in the research both through data forms 
and a scatter gram related to their professional focus and experience levels. 

The primary form used to collect data was called New Learning Data Form and was included in each attendee’s 
registration packet and available on tables at all scheduled conference sessions. The forms provided a place for 
participants to indicate the category, formal or informal, of their new learning, and also asked attendees to identify 
the specific setting in which the learning took place, such as symposium, hotel room, or at dinner. A reflective 
question prompted attendees to provide deeper information than typical evaluations: “Please tell us about any new 
learning you have experienced or any new questions that have occurred to you as a result of the conferenceP New 
Learning data collection boxes were placed throughout the conference venue in areas conducive to informal 
meetings and spaces for individual reflection. Researchers collected forms after each session and emptied collection 
boxes each night, dating, tabulating, and recording them. 

Each attendee had a unique identifying number to write at the top of each form submitted so that we could track 
submissions. Numbers were randomly assigned and could not be connected to participants’ identities. The numbers 
were used to analyze number of attendees who participated in the research and number of forms each submitted. 
Additionally, completion of forms qualified participants for daily drawings for prizes and their unique numbers 
served as ticket for that purpose. Over forty percent of attendees participated in the research through NL data sheets 
and their submissions ranged from one to fifty-two forms. In all, they submitted 1,000 forms. 

In addition to collecting data through forms, we conducted in-depth, semi-structured interviews on selected 
participants. Because we were creating an exploratory case study, we sought a diverse representation of 
interviewees. Our selection criteria included scholars, practitioners and students, as well as academy leaders, 
international members, and participants who fell into marginalized categories of attendees. 

In addition to the forms and interviews, we used participant observervation methods. Each member of the North 
Carolina State research team (four professors and four doctoral students) recorded observations and experiences in 
research journals. Eleven co-researchers, solicited by the AHRD board from a variety of institutions around the US, 
assisted with observations. All research assistants participated in both pre-conference training sessions and trial 
observations with feedback in preparation for their roles. Assigned to each scheduled conference program or 
presentation, participant observers recorded observations of sessions, reflections on the NL processes, and new 
learning they experienced in the context of the conference. 

To date, our findings have been reported in the Academy newsletter; at the International Conference of Human 
Resource Development, held in Leeds, UK; and in the Forum of the Human Resource Development Quarterly 
(Storberg-Walker, Wiessner & Chapman, in press). Multiple other research reports are in progress or planning. All 
of these efforts combine to make the data widely available to HRD scholars, practitioners, and other interested 
groups or individuals in order to benefit their work and to provide access to the divergent perspectives and 
experiences of conference attendees. Making NL available is a critical component of the methodology. 

Conclusions and Recommendations 

Traditional conference evaluation is usually program evaluation or event evaluation related to event planning and 
facilitation. New Learning synthesized that focus and examined ways in which the program or event facilitated or 
hindered learning while identifying and capturing what was learned. In other words, NL was about learning and 
knowledge generation, and not only about event planning and management. NL did collect data related to the 
conference venue, schedule, organization, processes and other dimensions, and these submissions can be useful to 
conference coordinators. These typical evaluative measures comprised a small component of the overall process. 

Not surprisingly, conference attendees demonstrated unfamiliarity with the NL processes. We noticed many 
attendees referred to New Learning ’05 as a new form of evaluation; many called our data collection forms 
“evaluation forms”. While they seemed to embrace the process and value its potential, they did not know how to 
categorize it. The research team explained repeatedly that we were not engaged in evaluation, rather in identification 
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and affirmation of learning. We were not evaluating the conference, rather recording and codifying what happens to 
conference participants and measuring energy generated by the “coming together” of people. 

A few people used NL forms to make reactionary statements about presentations, presenters, and conference 
venues, as they would on typical conference evaluations. This propensity in itself was an interesting finding for us, 
with relevant implications for future research. We did not see this use as a negative outcome. We saw it as evidence 
that the approach was more inclusive and identified a broader range of issues than originally intended. In addition, 
we found many attendees valued the process’ structured time to write and reflect on experiences at the conference. 
This value was evidenced by many attendees’ desire to keep a copy of completed data sheets; indications to us that 
both individuals and the organization felt the processes added value. Consequently, we initiated use of carbon NL 
data forms for future NL studies. This adaptation further advances the purpose and vision of NL - to facilitate the 
creation and extension of knowledge at many levels in order to use it for future learning and development. 

Once the value of NL is perceived, participants were less concerned about labeling it than they were about 
benefiting from it. As one program coordinator and NL participant stated, “I don’t care what you call it; I’m getting 
better feedback on my program than I ever have. It may be about learning for you, but it’s an amazing way of doing 
evaluation for me. I finally know what they’re learning and if we’re doing what we intend.” 

Is New Learning the next generation of evaluation? We think so. Not only can New Learning assess congruence 
of intended outcomes with actual outcomes, it extends popular notions of evaluation. New Learning contributes to 
knowledge creation, and actually speeds knowledge generation by making what is learned at an event available to a 
larger audience. New Learning shortens the cycle of exposure to new ideas and theories, giving participants, both 
scholars and practitioners, more dynamic and responsive ways of learning and constructing knowledge. 

Contribution to New Knowledge in HRD 

This research advances the mission of the Academy of Human Resource Development in several ways. It 
encourages systematic study of the field by providing attendees with new tools, methodologies and way of thinking 
about evaluation. It shows how knowledge can be generated and disseminated through publications, partnerships, 
contacts and especially, the synergy of group assembly. And, it helps attendees examine how engagement with 
scholars, practitioners and peers can generate new knowledge and further advancement of the HRD field. 
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